Cross-media Retrieval Based on CSRN Clustering

A novel cross-media retrieval methodology is proposed to help user more accurately and naturally describe their requirement and gain response. The first problem to be solved in this paper is how to build the bridge among heterogeneous information. An efficient approach is adopted to mine the semantic relationship among different multimedia data. We propose the scheme, called cross-media semantic relationship network (CSRN), to store the cross-media relationship knowledge, which is constructed by mining various potential relation information in the Internet. By hierarchical fuzzy clustering based on semantic relationship against CSRN, we obtain semantic vector bundle which could gather different modalities but same semantic of diverse media feature vectors. Furthermore, dynamic linear hash index is used in these vectors to adapt to the flexible expanding or updating of CSRN and quickly retrieval based on multiple examples without the limit of media modality by hash intersection. We show through experiment based on different modalities or different numbers of examples that our approach can provide more flexible retrieval mode and more accurate retrieval results.